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ABSTRACT 



A method and apparatus for providing personalized supple- 
mental programming is provided. In a first embodiment, an 
apparatus comprising a first component for supplying 
supplemental programming, a second component operative 
to receive primary programming from an external source, 
and a processor agent coupled to the first and the second 
components. The processor agent being operative to evalu- 
ate the received primary programming for predetermined 
content criteria, and to provide appropriate ones of the 
supplemental programming based on the results of the 
evaluation. 

34 Claims, 8 Drawing Sheets 
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METHOD AND APPARATUS FOR the prior art entertainment systems provide personalized 

PROVIDING PERSONALIZED supplemental programming to users of entertainment sys- 

SUPPLEMENTAL PROGRAMMING terns 

Thus, a need exists for a method and apparatus for 
RELATED APPLICATIONS 5 providing personalized supplemental programming content. 



This non-provisional application claims priority to pro- 
visional application No. 60/024,438 entitled Commentary 
Based on Content and Keywords, filed on Aug. 22, 1996, to 
Jean Goldschmidt, Tony Shah and David Hackson, and 
commonly assigned to Intel Corporation of Santa Clara, 
Calif. 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 15 
The present invention relates to the field of entertainment 

systems and, in particular, to a method and apparatus for 
providing personalized supplemental programming in enter- 
tainment systems, 

20 

2. Background Information 

Numerous advances have been made in recent years in the 
field of entertainment systems. Many providers of program- 
ming content now offer supplemental programming content 
designed to complement the normal/typical program infor- 25 
mation. For example, many television networks provide 
supplemental closed captioning of the audio information to 
allow people who are hearing impaired to view and enjoy the 
network's programming. The closed caption information is 
typically provided within line 21 of the vertical blanking 30 
interval (VBI) of the television signal. Another example of 
supplemental programming is the Supplemental Audio Pro- 
gramming (SAP) that is provided by many television 
networks, wherein the audio content of a broadcast is 
provided in an alternate language, or other supplemental 35 
audio content directed to the broadcast is provided. Many 
entertainment systems allow the system user to enable or 
disable such supplemental broadcast services. 

Other examples of supplemental programming exist 
throughout, for example, the Internet. Many of the web sites 40 
throughout the Internet, for example, provide supplemental 
programming information. Such information may be intro- 
duced by any of the intermediate routers between the user 
and the host web site. In particular, many of the Internet 
Service H Providers (ISP's) sell advertising space to local 45 
businesses, wherein the advertising will be displayed to all 
users browsing a particular web site, or web site genre. 
However, the web sites that provide such supplemental 
programming information often rely on the uniform resource 
locator (URL) of the accessed web site to determine which 50 
web site is being browsed and, consequently, which supple- 
mental information to provide. In other words, the ISP's do 
not evaluate the content of the information flow between the 
web site and the browser to determine what supplemental 
programming information to provide. In addition, this 55 
supplemental information is not personalized. Rather, it is 
common to a particular genre of web sites and provided to 
each user of the ISP accessing a particular genre of web 
sites. For example, those users viewing an automotive web 
site are likely to receive supplemental information regarding 6 q 
automotive products or services offered via the web, how- 
ever the same generic supplemental information is provided 
to all users of the web site. 

Insofar as all of the supplemental programming of prior 
art systems is supplied by the broadcast networks or in the 65 
stream of broadcast, and that such information is not truly 
personalized for the recipient of such inform nation, none of 



SUMMARY OF THE INVENTION 

In accordance with the teachings of the present invention, 
a method and apparatus for automatically providing person- 
alized supplemental programming is provided. In a first 
embodiment, an apparatus comprising a first component for 
supplying supplemental programming, a second component 
operative to receive primary programming from an external 
source, and a processor agent coupled to the first and the 
second components. The processor agent being operative to 
evaluate the received primary programming for predeter- 
mined content criteria, and to provide appropriate ones of 
the supplemental programming based on the results of the 
evaluation. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will be described by way of exem- 
plary embodiments, but not limitations, illustrated in the 
accompanying drawings in which like references denote 
similar elements, and in which: 

FIG. 1 is a block diagram illustrating a system providing 
personalized supplemental programming, incorporating the 
teachings of the present invention; 

FIG. 2 is a block diagram of a supplemental programming 
provider for evaluating primary programming content and 
providing personalized supplemental programming, in 
accordance with one embodiment of the present invention; 

FIG. 3 illustrates an example of a supplemental program- 
ming database suitable for use by the supplemental pro- 
gramming provider of FIG. 2, in accordance with one 
embodiment of the present invention; 

FIG. 4 illustrates an example of a primary programming 
database suitable for use by the supplemental programming 
provider of FIG. 2, in accordance with one embodiment of 
the present invention; 

FIG. 5 is a flow chart illustrating the method steps for 
providing personalized supplemental programming, in 
accordance with one embodiment of the present invention; 

FIG. 6 is a flow chart illustrating the method steps for 
evaluating primary programming for predetermined content 
criteria, in accordance with one embodiment of the present 
invention; 

FIG. 7 is a block diagram of a computer system incor- 
porating the teachings of the present invention; and 

FIG. 8 is a block diagram illustrating the software ele- 
ments in accordance with one embodiment of the present 
invention. 

DETAILED DESCRIPTION 

In the following description, for purposes of explanation, 
specific numbers, materials and configurations are set forth 
in order to provide a thorough understanding of the present 
invention. However, it will be apparent to one skilled in the 
art that the present invention may be practiced without the 
specific details. In other instances, well known features have 
been omitted or simplified for ease of explanation. 
Furthermore, for ease of understanding, certain method steps 
are delineated as separate steps, however, these separately 
delineated steps should not be construed as necessarily order 
dependent in their performance. 
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Turning to FIG. 1, a block diagram illustrating one programming provider 104. As illustrated in the functional 
example of a system incorporating the teachings of the block diagram of FIG. 2, example supplemental program- 
present invention is depicted. In accordance with the illus- ming provider 200 comprises commentary provider 206 
trated example embodiment, system 100 is depicted com- including keyword search engine 208 and content evaluator 
prising supplemental programming provider 104 which 5 214, supplemental programming database 210 (also referred 
receives primary programming (e.g., television to as a keyword specification module), primary program- 
programming, internet programming, radio programming, ming database 222 (also referred to as a program description 
satellite programming, etc.) via a number of alternative source, or an electronic program guide (EPG)), commentary 
mediums. For example, supplemental programming pro- generator module 216 and commentary delivery module 
vider 104 receives primary programming from wireline 10 218, communicatively coupled together as depicted, 
sources (cable television, cable radio, computer network \ Q tDe illustrated embodiment of FIG. 2, supplemental 
sources, telephone line, etc.) via line 102. Supplemental programming provider 200 receives primary programming 
programming provider 104 can also receive primary pro- v i a i me 202 at an input port which directs the primary 
gramming from wireless sources such as broadcast programming to commentary provider 206. As depicted, the 
television, broadcast radio, infrared computer networks, and 15 primary programming received via line 202 is intended to 
the like cumulatively depicted with signal 102', via appro- represent primary programming received via wireline 
priate antennae (not shown). sources (e.g., line 102 in system 100) and wireless sources 

As will be discussed in greater detail below, supplemental (e.g., signal 102' in system 100). In one embodiment, 
programming provider 104 evaluates primary programming commentary provider 206 receives primary programming 
information received from various sources (e.g., cable 102, 20 via line 202, and provides the primary programming to 
signal 102', an electronic program guide (EPG), or locally keyword search engine 210 and content evaluator 214. If the 
stored primary programming) against individual system user primary programming contains text information, such as the 
preferences for predetermined content criteria that corre- closed caption information found in the VBI of a television 
sponds with the individual system user preferences and signal, or computer data communications, keyword search 
personality, whereupon the supplemental programming pro- 2 s engine 208 evaluates the received text information for 
vider 104 outputs personalized supplemental programming content that satisfies the predetermined content criteria. In 
associated with the predetermined content criteria via line addition, the received primary programming is routed to 
106 to the individual elements of an entertainment system content evaluator 214, which evaluates the received primary 
(e.g., television system, audio system, computer system, programming for audio content and video/graphical content 
etc.), cumulatively identified within system 100 as enter- 30 which satisfies predetermined content criteria. Additionally, 
tainment system 108. That is to say, supplemental program- commentary provider 206 analyzes primary programming 
ming provider 104 continuously evaluates primary program- for trends associated with the predetermined content criteria, 
ming information for content that satisfies the predetermined In one embodiment, the trend information is stored locally 
content criteria found within a user profile for the current within commentary generator 206, while in alternate 
user of system 100, and, when the predetermined content 35 embodiments, the trend information is stored in an exter- 
criteria of the user profile is satisfied, supplemental pro- nally accessible storage location. It should be understood 
gramming provider 104 outputs supplemental programming that alternate embodiments of supplemental program pro- 
associated with the predetermined content criteria. In addi- vider 200 may include a filter (or filters) in the front end of 
tion to the foregoing, supplemental programming provider supplemental program provider 200 to filter text information 
104 monitors trends in information obtained from primary 40 from received primary programming, providing only text 
programming to provide supplemental programming asso- information to keyword search engine 210, while the 
ciated with those trends, and provides a host of external remaining programming content (e.g., audio, video/ 
outputs, cumulatively identified by line 110 to control envi- graphical, etc.) is provided to content evaluator 214. One 
ronmental elements such as room lighting, temperature skilled in the art will appreciate that although commentary 
controls (i.e., thermostat), and the like. Thus, as will be 45 provider 206 is depicted in the illustrated embodiment as 
described in greater detail below, the supplemental program- including both keyword search engine 208 and content 
ming may be informative, educational, entertaining, and/or evaluator 214, alternate embodiments of the present inven- 
program enhancing. tion may be practiced with only the keyword search engine 

In one embodiment, supplemental programming provider 208, or the content evaluator 214, without deviating from the 
104 may be a computer system incorporated with the 50 spirit and scope of the present invention, 
teachings of the present invention, as will be discussed l n one embodiment of the present invention, the prede- 
further with respect to FIG. 7, below. In another termined content criteria employed by keyword search 
embodiment, supplemental programming provider 104 may engine 208 to evaluate the received primary programming is 
be a "set-top" box endowed with the necessary processing saved in user profiles in supplemental programming data- 
power and incorporated with the teachings of the present 55 base 210. In one embodiment, supplemental programming 
invention. Alternatively, supplemental programming pro- database 210 includes a list of predetermined content criteria 
vider 104 may well be incorporated into individual elements (e.g., keywords, audio or visual) with corresponding supple- 
within system 100 (e.g., television system, stereo tuner, or mental programming associated with each of the predeter- 
computer network). mined content criteria entries, for each of a plurality of 

Having introduced the concept of the innovative supple- 60 system users. In alternate embodiments, supplemental pro- 
mental programming provider 104 above within the context gramming database 210 may well include additional user 
of system 100, FIG. 2 provides a block diagram which preference/profile information associated with the preferred 
illustrates one example architecture of supplemental pro- system environment for the corresponding system user. In 
gramming provider 200 suitable for use in system 100, accordance with the illustrated embodiment, the predeter- 
incorporating the teachings of the present invention. In one 65 mined content criteria may be default settings provided by 
embodiment, supplemental programming provider 200 is the manufacturer, or user defined content criteria entered in 
beneficially integrated into system 100 as supplemental user profiles of the supplemental programming database 210 
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by corresponding individual users of supplemental program- 
ming provider 200. Jumping ahead to FIG. 3, one example 
of a supplemental programming database (i.e., supplemental 
programming database 300) is illustrated. 

As depicted in the illustrated example embodiment of 
FIG. 3, supplemental programming database 300 including 
the individual user profiles is a three-dimensional relational 
database. In the illustrated embodiment, the y-axis is char- 
acterized by identifying the genre 302 of the predetermined 
content criteria and the associated supplemental program- 
ming. Those skilled in the art will appreciate that genre 
information 302 is but one of a number of alternative means 
by which the information within supplemental programming 
database 300 may be organized. Thus, a number of suitable 
alternative approaches exist for organizing the information 
of supplemental programming database 310, without depart- 
ing from the spirit or scope of the present invention. The 
x-axis is characterized by default content criteria 304, and 
user specified content criteria in the form of text 306, 
video/graphical 308 and audio 310 content. Corresponding 
supplemental programming is provided in the form of text 
312, video/graphical 314 and audio 316 supplemental pro- 
gramming. Moreover, audio supplemental programming 
316 may be generated in a number of alternative "person- 
alities" (e.g., Beavis, John Wayne, excited person, etc.) 
dictated by the personality index of column 318. In addition, 
personality index 318 may be relied upon to produce supple- 
mental programming corresponding to the personality of the 
user. 

As depicted in the illustrated example of FIG. 3, the z-axis 
of supplemental programming database 300 provides mul- 
tiple "pages" of the x- and y-axes (user profile) correspond- 
ing to a plurality of system users of system 100. In an 
alternate embodiment, supplemental programming database 
300 may simply contain a list of keywords with a corre- 
sponding list of supplemental programming responses. In 
yet another embodiment, supplemental programming data- 
base may simply include a list of keywords, relying on 
commentary provider 206 to store and determine which 
supplemental programming response is appropriate when 
the predetermined content criteria has been satisfied. 

As described within the context of the present invention, 
supplemental programming is intended to include a wide 
variety of content and media. In one embodiment, the 
content of the supplemental programming may be informa- 
tive (e.g., a definition of a spoken word, or a translation of 
a foreign language), or entertaining (e.g., a comment on a 
particular celebrity). In one embodiment, supplemental pro- 
gramming may include audio commentary on the received 
primary programming, video clips interjected in a pop-up 
window of, for example, a television or monitor, or text 
displayed on a television or computer terminal in response 
to received primary programming. In accordance with the 
illustrated embodiment, supplemental programming data- 
base 210 may be preloaded with a number of predetermined 
content criteria (e.g., default content criteria) with corre- 
sponding supplemental programming. In addition, supple- 
mental programming provider 200 provides a user interface 
which allows a user of the system to enter personalized 
content criteria (e.g., user specified content criteria) corre- 
sponding to a wide variety of media, and the associated 
supplemental programming corresponding to the predeter- 
mined content criteria. Those skilled in the art will appre- 
ciate that although supplemental programming database 300 
is depicted as a three-dimensional relational database, this is 
for ease of explanation only. That is, databases of greater or 
lessor complexity may be beneficially substituted for supple- 
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mental programming database 300, with a corresponding 
effect on the amount and complexity of the information 
contained therein. 

In addition to evaluating the received primary program- 

5 ming for the predetermined content criteria of supplemental 
programming database 210, in one embodiment keyword 
search engine 208 also searches primary programming data- 
base 222 for programming description information that 
satisfies the predetermined content criteria found in supple - 

10 mental programming database 210. Primary programming 
database 222 contains program description information on 
the primary programming available from a wide variety of 
sources. In one embodiment, for example, primary program- 
ming database 222 is a localized version of an on-line 

15 programming guide (e.g., an electronic program guide 
(EPG)), and the program description information is a sum- 
mation of the programming available from a particular 
source at a particular time. One example of a primary 
programming database is illustrated in FIG. 4. 

20 In the illustrated embodiment of FIG. 4, primary pro- 
gramming database 400 is a three-dimensional database, 
with primary programming referenced by time of day on the 
x-axis, primary programming source indicated on the y-axis, 
organized by the day of the week and month in the z-axis. 

25 In the illustrated embodiment, primary programming data- 
base 400 is updated on a monthly basis, however those 
skilled in the art will recognize that primary programming 
database 400 may be updated more frequently. In the illus- 
trated example of FIG. 4, a 24-hour day is divided into 

30 half-hour segments 404, 406, etc. In one embodiment, the 
primary programming information corresponding to the 
source of the primary programming and the time of day 
contains primary programming description information in 
the form of text. Accordingly, in the context of FIG. 2, 

35 keyword search engine 208 searches the primary program- 
ming database 222 corresponding to the time/source of the 
primary programming for text that satisfies the predeter- 
mined content criteria of supplemental programming data- 
base 210. Although primary programming database 222 is 

40 integrated with supplemental programming provider 200 in 
the example embodiment of FIG. 2, those skilled in the art 
will appreciate that primary programming database 222 may 
well be an external source of program description 
information, accessible by supplemental programming pro- 

45 vider 200 as necessary. 

Continuing with FIG, 2, supplemental programming pro- 
vider 200 also includes content evaluator 214 for evaluating 
the audio components and video/graphical components of 
the primary programming, an EPG, or stored programming 

50 for the predetermined content criteria associated with the 
user profiles stored in supplemental programming database 
210. In particular, content evaluator 214 analyzes audio 
input and video/graphical input for matches against the 
audio and/or video/graphical predetermined content criteria 

55 (e.g., found in columns 316 and 314 respectively) of supple- 
mental programming database 210. In one embodiment, for 
example, content evaluator 214 utilizes a speech comparator 
to evaluate the audio content of the received primary pro- 
gramming for audio content which satisfies the predeter- 

60 mined audio content criteria (e.g., as found in column 316) 
of supplemental programming database 210. In one 
embodiment, content evaluator 214 similarly utilizes a 
video/graphical comparator to evaluate the video/graphical 
images of the received primary programming for images 

65 which satisfy the predetermined video/graphical content 
criteria (e.g., as found in column 314) of supplemental 
programming database 210. Devices which analyze audio 
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information and devices which analyze video/graphical 
information are known to those skilled in the art, thus, 
further description is not necessary. 

With continued reference to the example embodiment of 
FIG. 2, if the predetermined content criteria is satisfied by 
the received primary programming, or corresponding 
description information from primary programming data- 
base 222, the appropriate supplemental programming is 
supplied to commentary generator 216 from the supplemen- 
tal programming database 210 by keyword search engine 
208 or content evaluator 214, respectively. In the illustrated 
embodiment, commentary generator 216 generates the 
supplemental programming from the input supplied by 
either keyword search engine 208 or content evaluator 214. 
For example, if the supplemental programming correspond- 
ing to the predetermined content criteria is comprised of 
audio information, commentary generator 216 generates an 
audio signal containing the supplemental programming 
content, in the personality identified by personality index 
318. In an alternate embodiment, keyword search engine 
208 or content evaluator 214 supply commentary generator 
216 with an index identifying a file which contains the 
supplemental programming. For example, with reference to 
FIG. 3, if keyword search engine 208 receives primary 
programming with the closed caption text reading "Michi- 
gan Touchdown", keyword search engine 208 provides 
commentary generator 216 with the content of the "Michi- 
gan Fight Song". Consequently, commentary generator 216 
generates audio supplemental programming from the Michi- 
gan. Fight Song input received from keyword search engine 
208. Thus, in one embodiment, supplemental programming 
provider 200 monitors primary programming, or the primary 
programming database for information corresponding to the 
predetermined content criteria, and provides the associated 
supplemental programming content identified in supplemen- 
tal programming database 210. 

In an alternate embodiment, commentary provider 206 
monitors primary programming for information trends, and 
provides supplemental programming in accordance with the 
analyzed trend information. For example, if from a user 
profile it is apparent that a user is interested in the perfor- 
mance of Intel stock, commentary provider will analyze the 
trend of the stock performance, providing supplemental 
programming as appropriate. In one embodiment, the trend 
information is stored locally within commentary provider 
206, and as primary programming is identified satisfying a 
predetermined content criteria associated with the perfor- 
mance of Intel stock, the corresponding supplemental pro- 
gramming is provided. In an alternate embodiment, the trend 
information may be stored remotely and accessed as needed 
by commentary provider 206. Another example wherein 
commentary provider 206 analyzes primary programming 
for trend information is, in accordance with the Michigan 
football game example above, commentary generator deter- 
mines in the third quarter that Michigan is down by a 
touchdown, commentary provider 206 generates audio 
supplemental programming to the effect of "Let's Go, 
Michigan", in a corresponding personality. 

In yet another embodiment, supplemental programming 
provider 200 analyzes the predetermined content criteria of 
a particular user profile to develop a "personality profile"for 
a user, and automatically generates supplemental program- 
ming in accordance with the generated personality profile. 
That is to say, supplemental programming provider 200 
produces supplemental programming based on a "personal- 
ity profile" identified in the user profile stored in supple- 
mental programming database 210. In one embodiment, for 
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example, a teenager is viewing a television show with a 
considerable amount of actio n — explosions, gunfire, etc. 
Based on the user profile associated with the teenager stored 
in supplemental programming database 210, supplemental 

5 programming provider 200 develops a personality profile for 
the teenager and identifies an affinity for such action. 
Consequently, based on the personality profile developed by 
supplemental programming provider 200, commentary gen- 
erator 216 generates a three-dimensional animated figure in 

10 a comer of the viewable screen of the television while 
providing the audio commentary "cool, man!", in accor- 
dance with the personality index associated with the teen- 
ager. Thus, rather than relying on pre-programmed supple- 
mental programming obtained from supplemental 

15 programming database 210, supplemental programming 
provider 200 is able to analyze the personality of the user 
(based on the user profile) to provide appropriate supple- 
mental programming. In alternate embodiments, the user 
may influence the personality profile as programming pref- 

20 erences change. 

With continued reference to FIG. 2, the supplemental 
programming generated by commentary generator 216 is 
supplied to a commentary delivery module (e.g., commen- 
tary delivery module 218). In one embodiment, commentary 

25 delivery module 218 synchronizes the supplemental pro- 
gramming with the primary programming for delivery to the 
appropriate system element (e.g., television, audio 
components, computer system, etc.). In one embodiment, 
commentary delivery module 218 supplies only the supple - 

30 mental programming to the appropriate system element 
(e.g., television, audio components), computer system, 
etc.). In an alternate embodiment, commentary generator 
216 supplies supplemental programming to the appropriate 
system element. 

35 In an alternate embodiment of supplemental program- 
ming provider 200, supplemental programming may be 
generated as a background process and supplied to com- 
mentary provider 206, which stores it for subsequent use as 
dictated by keyword search engine 208 or content evaluator 

40 214. In this alternate embodiment, commentary provider 206 
supplies the supplemental programming directly to com- 
mentary delivery module 218 without the intervening com- 
mentary generator module 216. 
Turning now to FIGS. 5 and 6, flow charts illustrating one 

45 embodiment of the method steps for automatically providing 
personalized supplemental programming are provided. For 
ease of explanation, and not limitation, the method of FIGS. 
5 and 6 will be developed in the context of system 100 
incorporating supplemental programming provider 200, 

50 supplemental programming database 300 and primary pro- 
gramming database 400. As illustrated, the method starts, 
step 500, wherein a user (e.g., user_l) begins using system 
100, thereby causing system 100 to begin to receive primary 
programming. Accordingly, supplemental program provider 

55 104 receives the primary programming at commentary pro- 
vider 206 via line 102, step 502. In step 504, the primary 
programming is evaluated for predetermined content crite- 
ria. Greater detail of such evaluation will be discussed in 
greater detail below, with reference to FIG. 6. In step 506, 

60 supplemental programming provider 104 determines 
whether the predetermined content criteria has been satis- 
fied. If so, supplemental programming provider 104 pro- 
vides the appropriate system components) with supplemen- 
tal programming from supplemental programming database 

65 210 that is associated with the predetermined content criteria 
satisfied, step 508. In an alternate embodiment, upon satis- 
fying the predetermined content criteria, supplemental pro- 
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gramming provider 104 issues supplemental programming high performance input/output (I/O) bus 706 and standard 
to the appropriate system component(s) that does not, I/O bus 708, as shown. Host bridge 710 couples processor 
necessarily, correspond to the primary programming, but is 702 to high performance I/O bus 706, whereas I/O bus 
randomly generated. Having provided the supplemental bridge 712 couples high performance I/O bus 706 to stan- 
programming in step 508 or, if the predetermined content 5 dard I/O bus 708. Coupled to high performance I/O bus 706 
criteria was not satisfied in step 506, the process loops back are network/communication interface 724, system memory 
to step 502. The method continues until the system is turned 714, audio/video interface board 730, supplemental pro- 
off, or the user disables the supplemental program provider. gramming provider 732 and video memory 716. In turn, 
As alluded to above, FIG. 6 is a flow chart illustrating the display device 718 is coupled to video memory 716. 
method steps for evaluating the primary programming and/ 10 Coupled to standard I/O bus 708 are mass storage device 720 
or programming description information obtained from pri- keyboard and pointing device 722, and I/O ports 726. In one 
mary programming database 222 for predetermined content embodiment, keyboard and pointing device are coupled to 
criteria (i.e., step 504), in accordance with one embodiment standard I/O bus 708 with a serial communication interface 
of the present invention. In accordance with the illustrated cable, while in alternate embodiments it may be communi- 
example embodiment, keyword search engine 208 accesses 15 catively coupled with an infrared (IR) interface or a radio- 
primary programming database for description information frequency (RF) interface. 

corresponding to the time/source of the received primary With continued reference to FIG. 7, elements 702-730 

programming, step 602. In step 604, commentary provider perform their conventional functions as known in the art. In 

206 accesses the predetermined content criteria of the particular, network/communication interface 724 is used to 

supplemental programming database 210 for use by key- 2 o provide communication between system 700 and any of a 

word search engine 208 and content e valuator 214. Keyword wide range of conventional networks, such as Ethernet, 

search engine 208 evaluates the textual information from the token ring,he Internet, etc. Similarly, audio/video interface 

received primary programming and/or the description infor- board 730 is used to receive broadcast communications from 

mation from primary programming database 222, against the any of a wide range of conventional broadcast media such as 

predetermined content criteria available in supplemental 2 $ RF broadcasts, satellite broadcasts, cable broadcasts, etc. 

programming database 210, step 606. Similarly, content Mass storage device 720 is used to provide permanent 

evaluator 214 evaluates the audio content and the video/ storage for the data and programming instructions to imple- 

graphical content of the received primary programming ment the above described functions, whereas system 

against the predetermined content criteria available in memory 714 is used to provide temporary storage for the 

supplemental programming database 210, step 608. In one 30 data and programming instructions when executed by pro- 

embodiment, the evaluation of criteria in the selection of cessor 702. I/O ports 726 are one or more serial and/or 

content is dependent on the user profile and the content parallel communication ports used to provide communica- 

personality. That is, if the primary programming is in lion between additional peripheral devices which may be 

accordance with the "personality profile" for a current coupled to system 700 (e.g., stereo, speakers, etc.). 

system user, supplemental programming provider 200 pro- 35 Collectively, the elements coupled to system 700 are 

vides supplemental programming associated with the pri- intended to represent a broad category of hardware systems, 

mary programming. In accordance with the illustrated including but not limited to general purpose computer 

example embodiment, if either keyword search engine 208 systems based on the Pentium® processor, the Pentium® 

or content evaluator 214 identifies primary programming Pro processor, or the Pentium® II processor commonly 

that satisfies the predetermined content criteria, it supplies 40 available from Intel Corporation of Santa Clara, Calif, 

commentary generator 216 with the appropriate supplemen- i n one embodiment, supplemental programming provider 

tal programming, returning to step 506. 732 includes the elements of supplemental programming 

In one embodiment, except for the incorporated teachings provider 200, including the elements necessary for perform- 
of the present invention (described above), supplemental ing audio analysis and video/graphical analysis of received 
programming provider 104 is intended to represent a broad, 45 primary programming and/or program description informa- 
category of computer systems known in the art. An example tion obtained from a primary programming database. In an 
of such a computer system is a desktop computer system alternate embodiment, supplemental programming provider 
equipped with a high performance microprocessors), such 732 is comprised of commentary provider 206 and com- 
as the Pentium® processor, Pentium® Pro processor, or mentary generator module 216, while the supplemental 
Pentium® II processor manufactured by and commonly 50 programming database 210 and primary programming data- 
available from Intel Corporation of Santa Clara, Calif.; base 222 are stored on mass storage 720, or are remotely 
including any of a number of audio and video input and located and communicatively coupled to supplementary 
output peripherals/interfaces for receiving, digitizing, com- program provider 732 via network/communication interface 
pressing and decompressing audio and video signals. It is to 724. In one embodiment, supplemental programming pro- 
be appreciated that the housing size and design for supple- 55 vider 732 receives primary programming from network/ 
mental programming provider 104 may be altered, allowing communication interface 724 and/or audio/video tuner inter- 
it to better visually fit within system 100. Accordingly, face 730, evaluates the primary programming, and returns 
supplemental programming provider 104 may well be supplemental programming to the appropriate elements in 
embodied in a "set-top" box incorporated with the teachings accordance with the teachings above. In an alternate 
of the present invention. 60 embodiment, supplemental programming provider 732 

FIG. 7 provides a block diagram of a computer system receives primary programming via an antennae (not shown) 
(e.g., system 700) incorporated with the teachings of the coupled to I/O ports 726, supplying supplemental program- 
present invention. In one embodiment, system 700 is supple- ming to network/communication interface 724 and/or audio/ 
mental programming provider 104 of FIG. 1. In the iUus- video tuner interface 730. In an alternate embodiment, 
trated embodiment, system 700 includes at least one pro- 65 supplemental programming provider 732 receives primary 
cessor (e.g., processor 702) and cache memory 704 coupled programming via I/O ports 726 and supplies supplemental 
to each other as shown. Additionally, system 700 includes programming to network/communication interface 724 and/ 
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or audio/video tuner interface 730, in accordance with the information obtained from primary programming database 

teachings above. 816 against the predetermined content criteria stored in 

It is to be appreciated that various components of system supplemental programming database 814. In the illustrated 
700 may be re-arranged. For example, cache 704 may be example embodiment, keyword search engine 808 and con- 
on-chip with processor 702. Alternatively, cache 704 and 5 tent evaluator module 810 of commentary provider 806 
processor 702 may be packed together as a "processor perform such evaluation. In one embodiment, upon receiv- 
module", with processor 702 being referred to as the "pro- ing a trigger from commentary provider 806 that the pro- 
cessor core". Furthermore, mass storage device 720, key- determined content criteria has been satisfied, commentary 
board and pointing device 722, and/or display device 718 generator module 812 generates supplemental programming 
and video memory 716 may not be included in system 700. 30 associated with the satisfied predetermined content criteria. 
Additionally, the peripheral devices shown coupled to stan- In an alternate embodiment, commentary generator module 
dard I/O bus 708 may, in alternate embodiments, be coupled 812 generates supplemental programming as a background 
to high performance I/O bus 706; in addition, in some process for use by commentary provider 806 when the 
implementations only a single bus may exist with the predetermined content criteria has been satisfied. In either 
components of system 700 being coupled to the single bus. 35 instance, supplemental programming is supplied to com- 
Furthermore, additional components may be included in mentary delivery module 813 for delivery to supplemental 
system 700, such as additional processors, storage devices, programming application 802 and appropriate system ele- 
or memories. ments via the operating system 820 and basic input/output 

In one embodiment, rather than including a separate system (BIOS) 822, which will be discussed in greater detail 

supplemental programming provider 732, the innovative 2 o below. In one embodiment, commentary delivery module 

features of the present invention discussed above may be 813 receives the supplemental programming as well as the 

implemented as a series of software routines run by system primary programming for delivery to the appropriate system 

700 of FIG. 7. These software routines run a plurality or elements. 

series of instructions to be executed by a processor, such as As alluded to above, BIOS 822 provides an interface 

processor 702 in system 700. Initially, the series of instruc- 2 5 between operating system 820 and the various I/O devices 

lions are stored on a storage device, such as mass storage coupled to the hardware system. Operating system 820 is a 

device 720. It is to be appreciated that the series of instruc- software service which provides an interface between BIOS 

tions may be stored on any conventional storage device, 822 and supplemental programming agent 804 as well as 

such as a diskette, CD ROM, magnetic tape, digital versatile other software applications, if any, being executed by a 

disk (DVD) (also referred to as a digital video disk), laser 30 computer system within which the present invention is 

disk, ROM, flash memory, etc. It is also to be appreciated practiced (e.g., system 700). C. Operating system 820 pro- 

that the series of instructions need not be stored locally, and vides an interface, such as a graphical user interface (GUI), 

could be received from a remote storage device, such as a between the user and the system controller. According to one 

server on a network, via network/communication interface embodiment of the present invention, operating system 820 

724. The instructions are copied from the storage device, 35 is the Windows™ 95 operating system, available from 

such as mass storage device 720, into system memory 714 Microsoft Corporation of Redmond, Wash. However, it is to 

and then accessed and executed by processor 702. In one be appreciated that the present invention may be used with 

embodiment, these software routines are written in the C++ any other conventional operating system, such as, for 

programming language. It is to be appreciated, however, that example, other versions of Microsoft Windows™ (for 

these routines may be implemented in any of a wide variety 40 example, Windows™ 3.0, Windows™ 3.1, Windows™ NT, 

of programming languages. In alternate embodiments, the or Windows™ CE), Microsoft DOS, OS/2, available from 

present invention may be implemented in discrete hardware International Business Machines Corporation of Armonk, 

or firmware. For example, an application specific integrated N.Y., the Apple Macintosh Operating System, available 

circuit (ASIC) could be programmed with the above from Apple Computer Incorporated of Cupertino, Calif., the 

described functions of the present invention. 45 NeXTSTEP® operating system available from Apple Com- 

FIG. 8 is a block diagram illustrating the software ele- puter Incorporated, or the UNIX operating system, available 

ments according to one embodiment of the present inven- from Santa Cruz Operations of Santa Cruz, Calif, 

tion. In particular software architecture 800 is shown com- Thus, the present invention automatically provides per- 

prising supplemental programming application 802, sonalized supplemental programming in response to primary 

supplemental programming agent 804 with associated com- 50 programming that satisfies predetermined content criteria, 

mentary provider 806 including keyword search engine 808 As discussed above, the present invention advantageously 

and content evaluator module 810, commentary generator monitors and evaluates received primary programming for 

module 812 and commentary delivery module 814, and an predetermined content criteria, and automatically supplies 

operating system 820 with associated drivers. In particular, personalized supplemental programming when such prede- 

supple mental programming application 802 interfaces with 55 termined content criteria is satisfied, 

supplemental programming agent 804 and provides the user While the method and apparatus of the present invention 

interface for supplemental programming provider 104 of has been described in terms of the above illustrated 

FIG. 1. embodiments, those skilled in the art will recognize that the 

In one embodiment, supplemental programming agent invention is not limited to the embodiments so described. 

804 is coupled to and able to obtain information from 60 The present invention can be practiced with modification 

supplemental programming database 814, and primary pro- and alteration within the spirit and scope of the appended 

gramming database 816. In an alternate embodiment, claims. For example, although depicted as separate 

supplemental programming database 814 and/or primary components, supplemental program provider 104 may be 

programming database 816 are integrated modules of incorporated into any number of system components (e.g., a 

supplemental programming agent 804. In one embodiment, 65 television/monitor/video cassette recorder/playback device) 

commentary provider 806 performs the steps of evaluating within system 100. Further, supplemental program provider 

the primary programming, and the program description 104 may not include all of the elements of FIG. 2 or 5, or 
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may, alternatively, include additional elements. Accordingly, 
the description is to be regarded as illustrative instead of 
restrictive on the present invention. 

Thus, a method and apparatus for automatically providing 
supplemental programming has been described. 5 

We claim: 

1. A user entertainment system comprising: 

a first component of the user entertainment system opera- 
tive to supply supplemental programming; 

a second component of Lhe user entertainment system 10 
operative to supply primary programming from a 
broadcast receiver; and 

a processor agent of the user entertainment system, com- 
municatively coupled to the first and second 

components, operative to evaluate the primary program- 15 
ming for content that satisfies predetermined content 
criteria that are associated with supplemental 
programming, the content criteria corresponding to 
individual user preferences for a current user of the 
entertainment system and operative to provide person- 2Q 
alized programming when the content criteria arc 
satisfied, the personalized programming comprising the 
primary programming provided simultaneously with 
associated supplemental programming selected based 
on the satisfied content criteria, 

2. The entertainment system of claim 1, wherein the first 25 
component stores content criteria for at least a subset of 
entertainment system users. 

3. The apparatus of claim 2, wherein the entertainment 
system develops a personality profile for each of the at least 

a subset of entertainment system users based on at least the 30 
content criteria for the at least a subset of entertainment 
system users. 

4. The apparatus of claim 3, wherein the processor agent 
is operative to generate supplemental programming in accor- 
dance with the developed personality profile. 35 

5. The apparatus of claim 3, wherein the processor agent 
provides the selected supplemental programming in a per- 
sonality corresponding to the developed personality profile. 

6. The apparatus of claim 1, wherein the content criteria 
and associated supplemental programming is provided by a 4Q 
user of the entertainment system. 

7. The apparatus of claim 1, wherein the content criteria 
and associated supplemental programming is provided by 
the manufacturer of the entertainment system. 

8. The apparatus of claim 1, wherein the content criteria 45 
and associated supplemental programming are provided on 

a medium readable by the supplemental programing. 

9. A method comprising: 

(a) receiving primary programming from an external 
broadcast programing source at a user entertainment 50 
system; 

(b) evaluating the primary programming at the user enter- 
tainment system for content that satisfies predeter- 
mined content criteria that are associated with supple- 
mental programming, the content criteria 55 
corresponding to individual user preferences for a 
current user of the entertainment system; 

(c) providing personalized programing when the content 
criteria are satisfied, wherein the personalized program- 
ming comprises the primary programming provided go 
simultaneously with associated supplemental program- 
ming selected based on the satisfied content criteria. 

10. The method of claim 9, wherein evaluating the pri- 
mary programming comprises: 

(a) accessing a primary programming database to retrieve 65 
primary programming description information corre- 
sponding to the external source and time of day; 
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(b) searching a supplemental programming database cor- 
responding to the system user for predetermined con- 
tent criteria that match text components of the received 
primary programming or the retrieved description 
information; and 

(c) evaluating audio/video components of the received 
primary programming for audio/video information that 
matches predetermined audio/video content criteria 
stored in the supplemental programming database cor- 
responding to the system user. 

11. The method of claim 9, wherein the content criteria 
and associated supplemental programming is entered into 
the supplemental programming database by a system user. 

12. The method of claim 9, wherein the content criteria is 
default content criteria of the supplemental programming 
database and associated supplemental programming is 
default supplemental programming. 

13. A machine readable medium for operation on a user 
entertainment system having stored therein a plurality of 
machine executable instructions, said plurality of instruc- 
tions when executed by a user entertainment system, causing 
said user entertainment system to perform receiving primary 
programming at the user entertainment system from an 
external source of broadcast programming, evaluating the 
primary programming at the user entertainment system for 
content that satisfies predetermined content criteria that are 
associated with supplemental programming, and providing 
personalized programming when the content criteria are 
satisfied wherein the personalized programming comprises 
the primary programming provided simultaneously with 
associated supplemental programming selected based on the 
satisfied content criteria. 

14. The machine readable medium of claim 13, further 
comprising instructions for allowing a user to define the 
content criteria and the associated supplemental program- 
ming. 

15. The machine readable medium of claim 14, further 
comprising instructions for developing a personality profile 
for the user, based on the content criteria, and for indepen- 
dently generating supplemental programming based on and 
in accordance with the personality profile. 

16. The machine readable medium of claim 13, wherein 
the personalized programming is associated with the content 
criteria. 

17. A user entertainment system comprising: 

a first component of the user entertainment system opera- 
tive to supply supplemental programming; 

a second component of the user entertainment system 
operative to supply primary programming from a 
broadcast receiver; and 

a processor agent of the user entertainment system, com- 
municatively coupled to the first and second 
components, operative to evaluate the primary pro- 
gramming for content that satisfies predetermined con- 
tent criteria that are associated with supplemental 
programming, the content criteria corresponding to 
individual user preferences for a content user of the 
entertainment system and operative to provide person- 
alized programming when the content criteria arc 
satisfied, the personalized programming comprising the 
primary programming superimposed by associated 
supplemental programming selected based on the sat- 
isfied content criteria. 

18. The apparatus of claim 17, wherein the entertainment 
system develops a personality profile for a subset of entice- 
ment system users based on content criteria for the subset of 
entertainment system users. 
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19. The apparatus of claim 17, wherein the content criteria 
and associated supplemental programming are provided by 
a user of the entertainment system. 

20. A method comprising: 

receiving primary programming at a user entertainment 5 
system from an external broadcast programming 
source; 

evaluating the primary programming at the user enter- 
tainment system for content that satisfies predeter- 
mined content criteria that are associated with supple- 30 
mental programming, the content criteria 
corresponding to individual user preferences for a 
current user of the system; 

a processor agent of the user entertainment system, com- 
municatively personalized programming comprising 1S 
the primary programming superimposed by associated 
supplemental programming selected based on the sat- 
isfied content criteria. 

21. The method of claim 20, further comprising develop- 
ing a personality profile for the user, based on the content 20 
criteria; and independently generating supplemental pro- 
gramming based on and in accordance with the personality 
profile. 

22. The method of claim 20, wherein the personalized 
programming is associated with the content criteria. 25 

23. A machine readable medium having stored thereon a 
plurality of machine, executable instructions, said plurality 
of instructions when executed by a user entertainment 
system, causing said user entertainment system to perform: 

receiving primary programming at the user entertainment 30 
system from an external broadcast programming 
source; 

evaluating the primary programming at the user enter- 
tainment system for content that satisfies predeter- 
mined content criteria that are associated with supple- 35 
mental programming, the content criteria 
corresponding to individual user preferences for a 
current user of the system; 

providing personalized programming when the content 
criteria are satisfied, the personalized programming 40 
comprising the primary programming superimposed by 
associated supplemental programming selected based 
on the satisfied content criteria. 

24. The machine readable medium of claim 23, further 
comprising instructions for allowing a user to define the 45 
content criteria and the associated supplemental program- 
ming. 

25. The machine readable medium of claim 23, further 
comprising instructions for developing a personality profile 
for the user, based on the content criteria, and for indepen- 50 
dently generating supplemental programming based on and 

in accordance with the personality profile. 

26. A user entertainment system comprising: 

a first component of the user entertainment system opera- 
tive to supply supplemental programming; 55 

a second component of the user entertainment system 
operative to supply primary pro gaming from a broad- 
cast receiver; and 

a processor agent of the user entertainment system, com- 
municatively coupled to the first aid second 60 
components, operative to evaluate the primary pro- 
graming for content that satisfies predetermined con- 
tent criteria that are associated with supplemental 
programming, the content criteria corresponding to 
individual user preferences for a current user of the user 65 
entertainment system and operative to provide person- 
alized programming when the content criteria are 



satisfied, the personalized programming comprising the 
primary programming and associated supplemental 
programming selected based on the satisfied content 
criteria, at least a portion of the selected supplemental 
programming replacing a component of the primary 
programming. 

27. The apparatus of claim 26, wherein the user enter- 
tainment system develops a personality profile for a subset 
of entertainment system users based on content criteria for 
the subset of entertainment system users. 

28. The apparatus of claim 26, wherein the content criteria 
and associated supplemental programming is provided by a 
user of the entertainment system. 

29. A method comprising: 

receiving primary programming at a user entertainment 
system from an external broadcast programming 
source; evaluating the primary programming at the user 
entertainment system for content that satisfies prede- 
termined content criteria that are associated with 
supplemental programming; 

providing personalized programming when the content 
criteria are satisfied, the personalized programming 
comprising the primary programming and associated 
supplemental programming selected based on the sat- 
isfied content criteria, at least a portion of the selected 
supplemental programming replacing a component of 
the primary programming. 

30. The method of claim 29, further comprising develop- 
ing a personality profile for the user, based on the content 
criteria; and independently generating supplemental pro- 
gramming based on and in accordance with the personality 
profile. 

31. The method of claim 29, wherein the content criteria 
and associated supplemental programming are provided by 
a user. 

32. A machine readable medium having stored thereon a 
plurality of machine executable instructions, said plurality 
of instructions when executed by a user entertainment 
system, causing said user entertainment system to perform: 

receiving primary programming at the user entertainment 
system from an external broadcast programming 
source; 

evaluating the primary programming at the user enter- 
tainment system for content that satisfies predeter- 
mined content criteria that are associated with supple- 
mental programming; 

providing personalized programming at the user enter- 
tainment system when the content criteria are satisfied, 
the personalized programming comprising the primary 
programming and supplemental programming selected 
based on the satisfied content criteria, at least a portion 
of the selected supplemental programming replacing a 
component of the primary programming. 

33. The machine readable medium of claim 32, further 
comprising instructions for allowing a user to define the 
content criteria and the associated supplemental program- 
ming. 

34. The machine readable medium of claim 32, further 
comprising instructions for developing a personality profile 
for the user, based on the content criteria, and for indepen- 
dently generating supplemental programming based on and 
in accordance with the personality profile 

providing personalized programming when the content 
criteria are satisfied, the personalized programming 
comprising the primary programming superimposed by 
associated supplemental programing selected based on 
the satisfied content criteria. 
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